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LAYERED GRAPHICAL USER INTERFACE 



NOTICE OF COPYRIGHT 

A portion of the disclosure of this patent document contains material that is 
subject to copyright protection. The copyright owner has no objection to the facsimile 
reproduction by anyone of the patent document or the patent disclosure, as it appears in 
the Patent and Trademark Office patent file or records, but otherwise reserves all 
copyright rights whatsoever. 

FIELD OF THE INVENTION 

The present invention relates generally to methods of providing a user interface 
for directing task(s) using a processing system, and more particularly to methods of 
creating a graphical user interface having user control(s) through a layered graphic file. 

BACKGROUND 

Graphical user interfaces (GUIs) provide a picturesque display of computer 
output and controls for a user to direct various operations of software. GUIs allow for 
endless possibilities of creating a user-friendly environment in which to view and 
interact with computer systems. 

It is usually important for a user interface to have a look and feel that is 
comfortable for the user. For many application programs, e.g. word processors, 
database programs, Web browsers, development tools, editing programs, 
communication programs, etc., it is desirable for the user interface to be designed with a 
control panel that it provides for ease of use. For example, in editing applications 
simple GUIs are especially significant. 

Some editing applications involve the manipulation of time based streams of 
information by novice users to create presentations. These time-based streams of 
information may be in the form of full motion images, e.g. video and movies; audio, 
e.g. music, sound effects, and voice/narration; motion graphics, e.g. animation; still 
images, e.g. text, pictures; and combinations thereof. The information is usually 
comprised of a stream of individual units, e.g. frames or blocks. Frequently, this 
information is in a digital format and resides on any of a number of conventional 
electronic mediums. 
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Some prior editing systems include Studio DV from Pinnacle Systems Inc. of 
Santa Clara, California; Avid Cinema® from Avid Technology, Inc. of Tewksbury, 
Massachusetts; Adobe Premier® from Adobe of San Jose, California; Dazzle* Digital 
Video Creator™ from Dazzle Multimedia of Fremont, California; and DVgate™ from 
5 Sony Corporation of Tokyo, Japan. 

Digital editing is typically performed in an editing processing system and 
involves random retrieval of information from storage, communication of the 
information, e.g. display on a monitor, and then manipulation of the information to 
produce the desired presentation. The presentation often appears as a seamless flow of 
1 0 the edited information. 

Traditionally, editing of multimedia to produce presentations involved complex 
techniques performed by trained professionals. However, recently novice users have 
become interested in making presentations, as well. The processing of multimedia to 
produce sophisticated home movies, web sites, business presentations, marketing 
1 5 demonstrations, amateur movies, etc. has been gaining in popularity. 

Given the currently diverse level of skills among authors of presentations, it is 
desirable to provide processing systems that are easy and convenient to operate. Such 
systems should have simple user interface displays from which a user may work. 

In general, a user may direct software operations by employing tools, referred 
20 to as "control elements/' e.g. buttons, menus, windows, text, etc, that are provided by a 
computer system. Typically, at lease some of the control elements are displayed on the 
user interface. Control elements may be active in instructing the software to execute 
operations, such as output information, or may be passive to simply display useful 
information on the screen, such as windows showing stored data. Several attributes of 
25 control elements presented on the display, such as the control elements' presentation, 
behavior, function, etc., influence the overall convenience of the GUI 

Applications for the GUI often vary among users and in certain uses control 
elements may be more important than other controls.. For example, different GUFs 
may be used to generate presentations for business talks, home movies, academic 
30 lectures, etc. However, most prior systems have fixed user interfaces that waste 
precious screen real estate on control elements that are less important for a particular 
application. It would be useful, then, for a processing system to provide a user interface 
that may be redesigned to emphasize only the desired control elements and to show 
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them in a preferred manner. Designers of the GUI should be permitted to easily change 
the characteristics of the control elements. 

Previously, GUI's have been created by first drawing the look of an image, e.g. 
the control element, with a graphic file and then converting the drawn picture into an 

5 intermediary format containing details regarding the image. The intermediate format is 
used to add various aspects of a control element, such its function, behavior, location, 
and/or the like. Such an intermediate format is any resource or file that is 
complementary to the application or built into the application that is required to fully 
describe a control element on a GUI. The intermediate format is picked up by an 

10 application program and depicted on a display. Since the user interface is a 
representation of the intermediary format, the user interface may not be altered through 
the graphic file without reconverting the entire file to the intermediary format. Thus, a 
control element may not be altered without recompiling the complete GUI. 

Whenever any component of the GUI is changed, a designer of the user 

1 5 interface must patiently wait while for the entire interface to be reconverted. As a result, 
much time is wasted while the designer remains idle. The lack of immediate feedback is 
inconvenient, especially when multiple attempts are made at editing before the desired 
modifications are achieved. However, with these previous processing systems, the 
designer may not inspect even the most minor changes to determine if the edits are 

20 acceptable until after the intermediate format step is complete and then accessed by the 
application. By comparison, it would be beneficial for a system to permit the designer 
to immediately view any alterations made to the interface. Quick feedback of revisions 
speeds up the editing of the GUI and simplifies revisions. 

Another drawback with these previous methods is that some of the details of the 

25 GUI may become lost in the conversion of the graphics drawing into the intermediary 
format. By contrast, elimination of the intermediate format step reduces the risk of lost 
details. Furthermore, by avoiding the intermediate format step, there is one less process 
that may provide bugs in the production of a GUI. 

In light of the shortcomings of the various currently available systems, there is 

30 still a need for a quick and simple method of creating a GUI in a manner. In particular, 
there is an interest in a processing system that allows a designer to easily edit the 
display of control elements on a GUI. 

SUMMARY OF THE INVENTION 



Application 



004860.P2475 



In one embodiment of the present invention, a method for producing a graphical 
user interface having at least one control element for use in controlling software 
operations is provided. The method includes storing a graphic file that has at least one 
control object each organized in a separate layer. An application program to display a 

5 control element on the graphical user interface and access the graphic file is launched. 
The control element that is shown on the user interface has at least one attribute that is 
dictated by one of the control objects. This control element may be an edit control for 
manipulating a time based stream of information. 

In one aspect of the method the graphic file has a control object dictating one 

10 attribute of the control element and located in one layer as well as another control 
object in a separate layer dictating another attribute of the same control element. These 
control elements may share the same common route name and the layers may optionally 
be grouped together. 

The graphic file is often editable so that a customized user interface may be 

15 produced by adding, deleting or changing the control objects within the layers. The 
entire GUI need not be converted to an intermediate format for each modification. 
Thus, the revised GUI may be immediately displayed. At times, this manipulation does 
not effect other portions of the graphic file unless the layer is grouped with another 
layer. Some of the attributes that are defined by the control objects include 

20 appearance, location or size of the element on the user interface. The control objects 
may also dictate the element type, state, function or behavior in a particular 
environment. 

In order to generate the GUI, a processing system may be employed which has a 
processor for storing the graphic file that has the control object(s) in a separate layer(s), 
25 and for launching the application program that accesses the graphic file and displays the 
control element on the graphical user interface. In another embodiment of the 
processing system, there includes a means for storing the graphic file and launching the 
application program. A means for altering the attribute by a single control object may 
also be provided. 

30 According to still further embodiments, a computer readable medium may be 

provided that has stored therein a plurality of sequences of instructions, which, when 
executed by a processing system, cause the processor to perform a variety of processes 
to related to producing the GUI with at least one control element. The instructions 
command the processor to store the graphic file and execute the application program. In 
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some cases, the control element is an edit control to manipulate a time-based stream of 
information. Of course, other embodiments may provide only the instructions 
themselves or methods of performing these steps. 

Other features and advantages of these and other embodiments are discussed in 
detail below. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention is illustrated by way of example, and not limitation, in the 
figures of the accompanying drawings in which: 

Figure 1 illustrates one embodiment of a working environment in which 
5 information may be processed, in accordance with the teachings presented herein. 

Figure 2 is a block diagram of one embodiment of a processing system, 
configured in accordance with the teachings presented herein. 

Figure 3A - 3C illustrates examples of various areas of storage, wherein Figure 
3A shows one embodiment of an information storage area having clips of time based 
10 streams of information, Figure 3B shows one embodiment of a reference storage area 
and Figure 3C depicts one presentation storage area during generating of a 
presentation. 

Figure 4 is an illustration of an image page having control objects in a graphic 
file, in accordance with the teachings presented herein. 
15 Figure 5 is an illustration of a list page having control objects in individual 

layers in a graphic file, in accordance with the teachings presented herein. 

Figure 6 illustrates a flow chart depicting one method for changing a user 
interface for editing according to the present invention. 

Figures 7A - 7C are examples of user interfaces for use in editing, according to 
20 one embodiment of the present invention, wherein Figure 7A is a GUI showing a 
reference icon, Figure 7B is a GUI showing controls for editing text into a presentation 
and Figure 7C is a GUI showing controls for editing transitions into a presentation. 

Figure 8 is a block diagram of a machine-readable medium storing executable 
code and/or other data to provide one or a combination of mechanisms for processing 
25 information, configured in accordance with one embodiment of the present invention. 
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DETAILED DESCRIPTION 

The methods described below are employed by a processing system to produce a 
layered graphical user interface with at least one control element defined by stored 
control objects. The control objects for each control element are stored in separate 
layers in a graphic file. The user interface is displayed by launching an application 
program that accesses the graphic file. The control objects are capable of controlling at 
least one attribute of a control element shown by the application program. The layered 
user interface, according to the present invention, permits a GUI designer to configure 
the control elements of the user interface to create a customized look and feel. 

In one embodiment, the processing system is an authoring tool to provide 
editing of a time based stream of information to generate a presentation. Through the 
graphical user interface, the editing processing system permits a user to easily control 
the editing process and select the information to be included in the presentation. The 
time based stream of information and related presentation created by use of the editing 
processing system may be of any form, such as full motion images with audio, full 
images without audio, audio only, still images, motion graphics or any combination 
thereof. 

Hardware Overview 
A processing system may be employed in a variety of contexts where a 
graphical user interface permits a user to dictate various tasks performed by the 
processing system. Figure 1 illustrates one exemplary working environment 10 in 
which a graphical user interface may facilitate a user to control software operations to 
process information, e.g. processing of a time based stream of information to produce a 
presentation. A processing system 12, according to one embodiment of the present 
invention, is in communication with an information source 80 that has information, at a 
capture port 14 (i.e. information collection interface) and optionally through an 
intermediate digital converter 82. Processing system 12 may read, copy, manipulate, 
store, delete, output, print and/or transfer information provided by source 80. An 
optional external player 84 receives communication from processing system 12, 
indirectly from a transfer port 16 and through pathway 86, and/or directly through a 
removable storage medium 18. Although Figure 1 demonstrates one layout of working 
environment 10, the scope of the present invention anticipates any number of 



Application 



7 



004860.P2475 



information sources and players, coupled and arranged in various fashions and 
configured in accordance herewith. Furthermore, the system for using a graphical user 
interface may be utilized without an information source, digital converter and/or 
external player. 

The information source 80 is any type of device that generates and transfers data 
or data signals related to the information. The information source 80 may be a 
photographic or full motion image capture interface such as a camera, camcorder, 
digital camera, etc.; a sound capture interface, such as a laser disc player (e.g. CD 
player), tape player/recorder, MP3 player, microphone, etc.; a keyboard; a scanner; a 
computer; a server; a computer program; the Internet, a sensor; any one of numerous 
available data collection devices; etc. 

The information source may generate digital data signals and be 
communicatively coupled directly to the processing system 12. Alternatively, the 
information source may generate signals in a format other than digital, e.g. analog 
signals. As shown in Figure 2, such non-digital data source may send signals to a 
digital converter 82 for changing the signals into a digital format prior to transfer of the 
data to the processing system 12. In another configuration, the information source 80 
resides in the system, such as a software file having data that is imported to an 
application program employed by the present invention. 

Communication between the information source 80 and the processing system 
12 is often through means of a FireWire® (from Apple Computer, Inc. of Cupertino, 
California), iLink or IEEE 1394 communication protocol. Where a FireWire brand 
protocol is employed, capture port 14 in the processing system may be a six-pin 
FireWire brand port. Also, a corresponding four-pin FireWire brand port may be 
present on the digital converter 82, where a digital converter is included, and/or on the 
information source 80. In this case, a six-pin-to-four-pin cable may be used to connect 
the ports of the digital converter 82 or information source 80 and processing system 12. 

Other communication schemes are also suitable for transfer of data from 
information source 80 and/or digital converter 82 to the processing system 12, such as 
an Ethernet connection (i.e., capture port 14 may be an Ethernet port), serial interfaces, 
parallel interfaces, RS422 and/or RS432 interfaces, Livewire interfaces, Appletalk 
busses, small computer system interfaces (SCSI), ATM busses and/or networks, token 
ring and/or other local area networks, universal serial buses (USB), PCI buses and 
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wireless (.e.g., infrared) connections, Internet connections, and other communication 
links for conveying the time based stream of information from an information source 80 
to the processing system 12. In addition, source 80 may store the information on a 
removable storage source, which is coupled to, e.g. inserted into, the processing system 
12 and in communication with the capture port 14. For example, the source 80 may be 
a tape, CD, hard drive, disc or other removable storage medium. 

The processing system 12 is any device configured to receive the information 
and manipulate the information, for example to generate a presentationThe processing 
system 12 may be a computer system, workstation, and the like. In one embodiment, 
the processing system 12 includes a platform 20, e.g. a personal computer (PC), such as 
a Macintosh® (from Apple Computer, Inc. of Cupertino, California), Windows®-based 
PC (from Microsoft Corporation of Redmond, Washington), or one of a wide variety of 
hardware platforms that runs the UNIX operating system or other operating systems. 
Often, the processing system 12 is configured to send the resulting information to an 
external player 84. 

The external player 84 may be an EP host, a computer, a personal digital 
assistance (PDA), a server, a tape/CD/MP3 player, or other device that is configured to 
receive the information from processing system 12 and to display the information. In 
one instance of a working environment, the external player 84 is the same device as the 
information source 80, such as a camcorder. In still another case, no external player is 
provided and the information is played on the processing system and/or projected from 
the processing system onto a medium, e.g. a screen. 

The external player 84 may receive information directly through removable 
storage 18 from the processing system 12. Removable storage 18 represents any of a 
number of detachable storage mediums, e.g. magnetic, optical, magneto-optical, tape, 
and/or other type of machine-readable medium or device for storing information. For 
example, the removable storage 18 may be a hard disk, a read-only or writeable optical 
CD (e.g. CD ROM, DVD), a disc, tape, etc. 

External player 84 may also receive presentation information indirectly from the 
processing system 12, such as through a pathway 86. The communication pathway 86 
may be through various networking mechanisms, such as a Fire Wire brand connection 
(iLink or IEEE 1394 connection), LAN, WAN, telephone line, serial line Internet 
protocol (SLIP), point-to-point protocol (PPP), an XDSL link, a satellite or other 
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wireless link, a cable modem, ATM network connection, an ISDN line, a DSL line, 
Ethernet, or other communication link between the processing system and external 
player. 

The processing system has components for handling information in various 
ways. The processing system 12 in Figure 2 includes an optional capture port 14 for 
acquiring the information, a storage 30 for holding information and control tools, a 
processor 26 for manipulating the information and performing other operations, an 
optional process manager 66 for instructing the processor on editing and storage of 
information, a display 70 for showing a user interface and possibly a transfer port 16 or 
removable storage 18 for sending the information so processed. Upon review of this 
specification, it will be appreciated by those skilled in the art that the components of 
processing system 12 may be connected in various ways in addition to those described 
herein. 

Furthermore, although Figure 2 shows these components in the context of an 
editing processing system for generating a presentation of a time based stream of 
information, it should be recognized that the processing system described below is only 
one example of a system that may use the GUI produced by the methods presented 
herein. The description of a processing system herein is not intended to limit the scope 
of the present invention to such a processing system and its components. Thus, the 
present invention anticipates the production and use of the GUI in a variety of other 
systems where a graphic file is directly accessed to generate the GUI with its various 
control elements. Other systems may employ the GUI of the present invention for 
controlling any software application, such as a word processor, database program, Web 
browser, development tool, editing program, communication program, and the like. 

Where an editing processing system is used with the GUI, at least one time 
based stream of information is assembled into a presentation. For example, a video may 
be edited and combined with audio, text, effects and/or graphics to create a 
presentation. A "presentation" is any changed version of an original time-based stream 
of information or a modified copy of the original information. In addition, a 
"presentation" also may further be defined by at least one instruction for representing 
the original information in a changed form. In such an instruction-based presentation, 
the information is communicated in the manner according to the instruction. For 
instance, a software program may include directions on how a digital video should be 
played, where only certain frames of a video are chosen for display. 
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A portion or portions of sequential information, also referred to as a "clip," such 
as a continuum of segments, frame, block or footage, may be rearranged into a desired 
order. Editing may further entail removing portions of clips. During editing, particular 
"edit features" may be added to the presentation, such as transitions between clips, 
special effects (e.g., black and white, brightness, contrast, texture, echoes, compositing, 
blurring, etc.), text, graphics, black background, silent audio segments, combinations of 
different types of time-based streams of information, such as audio and still images, and 
the like. Various types of editing may be performed as new techniques for altering raw 
information are created. 

In general, editing processing systems require that the time based stream of 
information be captured from a media source and entered into the system. The 
information, as well as the newly created presentation, is stored within memory of the 
system. Typically, the presentation is in the form of references that correspond to the 
stored information. The references have instructions on handling the information. These 
editing processing systems permit the user to control what is included in the 
presentation by moving the in and out points in the references to indicate the segments 
of information that are to be accessed. Thus, a presentation may include instructions 
regarding the portion of the information in which output is to begin and the portion of 
the information where the output is to cease. For example, the references may direct the 
select portions of the information that are to be included in the presentation. The 
references also may include edit features to be added to the presentation. Through a 
user interface presented on a display, reference elements that represent the references in 
storage are manipulated. In these instruction-based editing processing systems a user's 
edit choices are embodied by the references. 

Now referring in more detail to the components shown in Figure 2, processing 
system 12 includes processor 26, which may represent one or more processors. Some 
exemplary processors are a Motorola Power PC processor, an Intel Pentium (or x86) 
processor, etc. The processor 26 may also be a microprocessor. The processor 26 runs 
an operating system and applications software that controls the operation of other 
system components. The processor may be configured to perform multitasking of 
several processes at the same time. The processor may perform the operations required 
to produce the GUI, such as storing a graphic file having at least one control object and 
launching an application program to access the graphic file. 
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Storage 30 is coupled to processor 26 by a bus 22. The storage 30 may be a 
dynamic random access memory (DRAM) and/or may include static RAM (SRAM). 
Typically the storage is a hard disk drive, but may alternatively be another magnetic 
disk, a magneto-optical disk or other read/write device. The processor may also be 
5 coupled to other types of alternative storage areas that may be considered as part of the 
storage 30 or separate from the storage 30. Such alternative storage areas may include 
cache, flash memory, etc., and which may also be considered the removable storage 18. 
Usually storage 30 has a capacity in the megabyte to gigabyte range, and more 
typically, at least 64 megabytes capacity and can sustain throughput of at least 7 

10 megabytes per second of information. However, as technology advances, the capacity 
of the storage may increase. 

The storage 30, as used in a processing system, may contain information within 
files as well as executable code that may provide functionality for processing the 
information. For example, a time based stream of information may be stored within 

15 files in an information storage area 50. One exemplary information storage area 50 is 
depicted in Figure 3A as having a segment of a time based stream of information, clip 
A 100 with units (0 to 50), and another fragment, clip B 110 having units (0 to 30). 

A storage in an editing processing system also may have at least one and usually 
multiple references to the stored information. The reference includes data, e.g. 

20 instructions, on how the processor is to read and/or manipulate the stored information. 
Typically, a reference is an object that corresponds to an individual clip of information. 
The reference data include at least one in-point that specifies where the processor is to 
start accessing the stream of information and at least one out-point that identifies where 
to stop access of the information. 

25 The reference data may also contain instructions on adding particular edit 

features to a presentation that are in addition to the stored information and location data 
on where such edit features are to be presented in the time based stream of information. 
Some edit features are text, transitions, graphics, special effects (e.g. black and white, 
brightness, contrast, texture, echoes, compositing, blurring, etc.), black background, 

30 silent audio segments, overlays, three-dimensional images, audio, etc. In one 
configuration, edit features are provided by a plug-in or Active-X control and made 
available to an editing application program used herein. The edit features available for 
use with the present invention may be changed and new edit features added to the 
system. Edit features are any modifications that may be made to the units of the original 
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time based stream of information for use in the presentation, that exist with current 
editing technology or may be created in the future. 

The references to the stored information may reside in a reference storage area 
52. One embodiment of reference storage areas 52 representing edited references based 
5 on the information is shown in Figure 3B, where reference storage area 52 corresponds 
to the information of Figure 3 A with modified reference Al (0 to 15) 106 
corresponding to clip A, but only from units (0 to 15) 100, a copy reference A2 (0 to 
50) 104 and reference B (0 to 30) 112 corresponding to clip B units (0 to 30) 110. 

The stored references may be indexed in a reference database 32 within the 
10 storage 30. Each entry 34 in the database is to a reference and provides various data 
found within the reference, such as the associated clip 36, an in-point 38 and an out- 
point 40. For example, a reference (1) may indicate clip (A) with an in-point at frame 

mi 

(0) and out-point at frame (60). The database may also include other reference data that 
Zj assists in generating a presentation. 

^ 15 The presentation that is generated by an editing processing system, as 

yg represented by the references, may be stored in presentation storage area 54. The 

presentation storage area includes data related to when the information is to be 
^ displayed along a presentation time line 56. One presentation storage area 54 that draws 

Q upon the pool of reference data from Figure 3B is shown in Figure 3C and begin with 

JS: 20 reference Al (0 to 15) 106 corresponding to clip A units (0 to 15) 100 and immediately 
O followed by reference B 112 corresponding to clip B units (0 to 30) 110. 

The storage 30 also has data that is used to project the graphical user interface. 
A graphic file 56 is stored with at least one control object 60 having instructions for the 
control elements, that are displayed on the user interface. The graphic file 56 may be 
25 produced by any multiple layered-type computer program for creating and manipulating 
graphic images on a computer, where separate layers have control object(s), e.g. Adobe 
Photoshop®, by Adobe Systems Incorporated, located in San Jose, CA.; and Paint Shop 
Pro™, by Jasc Software, Inc., located in Eden Prairie, MN. 

The control object 60 in the graphic file 56 dictates the attribute(s) for visible 
30 control elements, i.e. images, displayed on a user interface for creating the presentation 
through the use of references. Active control elements may be used to control any 
software task and passive control elements may be used to output information or data. 
In an editing processing system, for example, least one control element is an edit 
control to allow a user to manipulate a time based stream of information and design a 
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presentation or otherwise control it. One control object may define one or several 
attributes for a given control element and different types of control objects may be in 
pages within the graphic file. 

Some of the control objects determine attributes that are related to the bits or 
5 pictures drawn from each layer in the graphic file to the user interface. Such attributes 
include the appearance of the control element, such as color, shape and text (font, size 
and/or wording). Other attributes may be the location of the element on the user 
interface and the size of the element. These picture-related control objects may be 
embodied in an image page 200 in the graphic file 56 as shown in Figure 4. The control 
10 objects 202 depict each control element as they would substantially appear on the user 
interface. Thus, the graphic file, through control objects stored in separate layers, 
contains the attributes required to display the entire GUI and there is no need to convert 
£ the images from a graphic file to an intermediate format. 

In order to change, add or delete an attribute, a control object 202 is selected 
Sf 15 from or added to the image page 200 and the particular attribute reconfigured using 
y3 commands as set forth by the graphic file program, e.g. through menu selection. For 

^ example, to relocate a control element on the user interface, the corresponding control 

^ object may be selected, dragged to new position and dropped into place, e.g. by use of a 

tj mouse. Similarly, to change the size of a control element, the associated control object 

m 20 may be selected and expanded or reduced, e.g. via cursor movement or menu selection. 
£3 In addition, to change the text appearing on a control element, the control object may be 

again selected and the text retyped, e.g. via keyboard input. 

Other attributes of control elements are behavior-related to determine the 
function that the control element represents. The control objects that correspond to 
25 control element behavior may be recorded in a layer list page 204 in the graphic file 56. 
One such layer list page 204 is shown in Figure 5. The page has multiple layers 208 of 
control objects 206 describing some typical behavior-related attributes, where usually 
each entry represents a single layer having a control object. One attribute typically 
defined in a layer list page is the name 210 of the control element, e.g. Play Full Screen, 
30 Title Color, Transition Popup Pane, Trash Can, etc. A control object 206 also may 
describe the type 212 of the corresponding control element, such as button, slider, static 
text, table pane, pop-up tab pane, etc. In addition, the state 214 of the element may be 
identified by a control object, especially where the associated control element has 
multiple states. Some such state attributes include pressed, up, down, disable, uproll, 
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etc. Several states may be defined for a particular control element by separate control 
objects for each state, where individual control objects for the same control element are 
located in separate layers, share a common root name and specify the characteristics of 
the element for each of the different states. For example, Play Full Screen may be a 
5 name for a control element assigned to several control objects for particular states of the 
Play Full Screen control element. Further to this example, a control object may dictate 
that when the Play Full Screen element is in the "up" state, the full screen view of a 
selection is played. The command or function 216 of the element is another common 
attribute identified by a control object. Examples of functions include playing the full 

10 screen, preview the title, etc. In addition, the behavior 218 of the element may be 
defined for a particular environment, such as the behavior in different resolutions in 
which the application program may be executed. A control object may describe the 
location of a control element at 800 x 600 resolution and the location at 1024 x 768. For 
example, a layer named "TitlePopupPane" may have a description, "anchor:br 

15 moverfixed resizerfixed" to specify that the TitlePopupPane is anchored to the bottom 
right corner of its parent pane, so if the parent pane is resized, the TitlePopupPane 
remains the same distance from the bottom right corner of the parent pane, as specified 
by the graphic file. Other objects may describe an element as "sticking" to a corner or 
side of a window; resize horizontally, vertically or in both directions; or stay centered. 

20 The attributes may also include any other feature that describes a given control element, 
some of which attributes may be specific to a particular control element. In addition to 
the methods described herein, the attributes may also be defined by various other means 
of drawing control objects to individual layers, as established by the particular graphic 
file employed. 

25 In order to alter the control elements (attribute) represented by control objects 

on the layer list page 204, the control object in a layer is reconfigured. The control 
object in a particular layer is selected and the description of the attribute of interest is 
changed, e.g. retyped via keyboard input or menu selection. For convenience, control 
objects in multiple layers may optionally be grouped together so that all objects of a 

30 particular group may be described as acting in the same manner or changes to one 
member of a group will be automatically transferred to the other group members. In 
addition, members of a group may be further subdivided into subgroups. Another type 
of connection is a link among layers through link control 220. Linked layers may be 
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placed in the same general location in the graphic file and attributes changes made to 
any single layer are applied to all associated links. 

Typically, each control object is contained in an individual layer in the graphic 
file. The layers permit the user interface to be worked on as if it was on a series of 
5 transparent overlays. Each control element of the user interface may be treated as an 
independent entity. Thus, any control element may be selected and changed without 
affecting the other control elements. A control element may have one or many 
associated control objects related to different states or aspect of the control element, e.g. 
1 to 100 layers may relate to a single control element. The order in which the layers are 

10 listed in the graphic file determines the order in which the control elements are written 
in the user interface. Usually, the interface is drawn by first retrieving the bottom layer 
and then proceeding to the top layer and the top of the list appears on the user interface 
in front of the layers at the bottom. 

According to the present invention, the graphic file is accessed directly by an 

15 application program 64, e.g. video editing software. The application program 64 
retrieves the drawn layer data to display the components of the user interface. The 
graphic file is thereby opened in its native file format. The graphic file may be 
maintained by an operating system as a file that is separate from the application 
program. By this direct access configuration, a GUI designer may easily change any 

20 aspect of the user interface at any time by altering the graphic file and redrawing 
individual layers contained therein. 

As shown by the flow chart in Figure 6, in order to change an attribute of a 
control element on the user interface, the corresponding control object in a particular 
layer in the graphic file is retrieved 222. The description of the attribute in the isolated 

25 control object is changed as desired 224 and saved 226, usually without affecting the 
control objects in any other layer unless the particular layer is grouped (or subgrouped) 
with or linked to other layers. The application program reads the revised graphic file 
228 and immediately depicts the new user interface 229 for viewing by the user. 

Active control elements, such as those displayed on the GUI, may be activated 

30 by the user through input/output device 92. In order to input commands and output 
responses, the processor 24 of the system for example the processing system 12 shown 
in Figure 2, is coupled by bus 22 to an input/output (I/O) control 90. The I/O control 90 
commands one or multiple I/O device(s) 92. Some conventional I/O devices are key 
boards, mouses/trackballs or other pointing devices, microphones, speakers, magnetic 
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disk drives, optical disk drives, printers, scanners, digital cameras, etc. There are a 
variety of interactive mechanisms to activate the control elements, such as keyboard 
input, voice command, mouse manipulation, text entry, pen-to-text data entry device, 
touch screen, network signals, satellite transmissions, preprogrammed triggers within 
the system, instructional input from other applications, etc. 

The control elements respond to activation by sending instructions to the 
processor. For example, a processing control element conveys directions to the 
processor regarding how a clip is to be treated or displayed. Each control element 
includes properties that describe how it will react when selected by the user or other 
means. 

The processing system may contain any number of different types of control 
elements. All of the control elements described herein are by way of example and are 
not intended to limit the choices that are or may become available in the art. 

Some forms of control elements that are visible on the screen are buttons, "pop- 
up" or pull-down menus, scroll bars, panes and iconic images. Other control elements 
may be invisible to the screen and thus are not included on the GUI Control elements 
that are not visible on the screen may be present on input/output devices, such as a 
keyboard that is in communication with the I/O control. For example, a space bar on a 
keyboard may also be depressed to start capture and depressed again to stop capture of 
the information on the fly while playing the time based information during capture 
mode. 

The table shown below describes other control elements that may be present on 
the keyboard of a processing system for manipulating a time-based stream of 
information. The items in the "action" column describe the command activated by 
depressing the keyboard button and the items in the "shortcut" column are the 
corresponding keyboard buttons. 
Keyboard shortcuts Navigation 
Action Shortcut 

Play/Stop and Start/Stop Capture Space Bar 
Play-head to Beginning of Movie Home 
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Play-head to End of Movie End 



Forward One Frame 



Forward Ten Frames 



Right Arrow 



Shift + Right Arrow 



Roll Play-head Forward Hold Down Right Arrow 



Search Forward (f forward) Command + ] 



Back One Frame 



Left Arrow 



Back Ten Frames 



Shift + Left Arrow 



yd 



Roll Play-head Backward Hold Down Left Arrow 



Search Backward (rewind) Command + [ 



Selection 
Action 

Multiple Selection 



Shortcut 



Shift + Click Item 



Multiple Selection (range) Click 1st item then Shift + Click last item 



Moving/Cropping 
Action 

Move Audio Clip 



Shortcut 



Click Clip + Left or Right Arrow 



Move Audio Clip Ten Click Clip, then Shift +Left or Right Arrow 
Frames 



Move Crop Marker Click Marker + Left or Right Arrow 
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Move Crop Marker Ten 
Frames 


Click Marker, 
Shift +Left or Right Arrow 


then 


Accept/Cancel 
Action 


Shortcut 




Accept Dialogs (OK) 


Return 




Cancel Dialogs 


Esc or Command + C 




Cancel Rendering 


Command + . (period) 




Volume 
Action 


Shortcut 




Increase System Volume 


Up Arrow 




Decrease System Volume 


Down Arrow 




Titling 
Action 


Shortcut 




Move Between Text Fields 


Tab 




File Menu Functions 
Action 


Shortcut 




New Project 


Command + N 




Open Project 


Command + O 




Save Project 


Command + S 
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Export Movie Command + E 

Save Frame As Command + F 

Import File Command + I 

Quit Command + Q 

Edit Menu Functions 

Action Shortcut 

Undo Command + Z 

Redo Shift + Command + Z 

Cut Command + X 

Copy Command + C 

Paste Command + V 

Crop Command + K 

Split Clip at Play-head Command -h T 

Select All Command + A 

Select None Command + D 
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Help Menu Functions 
Action Shortcut 



Imovie Help Command + ? or Help key 



To show the visible control elements, as shown in Figure 2, a display control 
74, e.g. video board, is coupled to the processor 26 through bus 22. The display control 
5 74 communicates with the display 70 in the processing system 12 to couple the display 
70 to the platform 20. Display 70 may be any one of a number of conventional display 
devices such as a liquid crystal display, cathode ray tube, plasma display, video display, 
or other type of display device. For sharp colors and grayscale, display 70 may be an 
SVGA monitor, with a resolution of 800 x 600 (26 dpi) or better, with an active area of 

10 at least 15 inches, and may be controlled using a true color video card. The visible 
control elements as well as other editing data may be depicted on a graphical user 
interface (GUI) that is shown on a display. 

The GUI used in generating a presentation may have numerous innovations that 
allow a user to easily make edits and create a presentation. An exemplary GUI 230 for 

1 5 displaying various data related to both the capture of time based streams of information 
and processing of the information is depicted variously in Figures 6A to 6C. The 
present invention frequently has a single GUI to display enabled, i.e. operable, capture 
information and enabled, i.e. operable, process information. Such a GUI is functional 
for both capture and edit procedures, thus avoiding the necessity of switching to 

20 multiple GUPs to view and work with information relating to capture and information 
relating to the processing of the presentation. Any convenient portions of the screen 
may be used to show the capture and process information without the user needing to 
toggle between screens. 

As shown in Figure 6A, the process information displayed on the GUI 230 may 

25 include at least one reference element 236, e.g. icon, with each reference element 236 
being associated with a reference in storage that has data for a clip of the time based 
information contained in storage 30. Typically, the reference element 236 is displayed 
as a "thumbnail" in the form of a still image of the first frame of the associated clip. 
Where the reference has an edit feature, a description or depiction of the edit feature 

30 may also be included on the reference element, such as a portrayal of the edit feature 
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350, or a particular shape, size or texture to indicate the edit feature. For example, a 
reference to a transition may be in the form of a box 352. 

Typically, control elements in the form of one or more windows are present in 
the GUI, which may be opened by processor 26 or by the user. One such user interface 
5 for use with an editing processing system illustrated in Figure 7A has several windows, 
such as a monitor window 246, time line 240 and edit box 268. The monitor window 
246 may be the viewing portion of the display that shows the edit output. Various other 
windows may be present on the display of the processing system to provide at least one 
control element. 

1 0 At any time during the edit process, the system may produce an edit output to 

reveal stored clips of the time based information for processing/editing e.g. the clip 
associated with a reference element may be viewed on the monitor window 246. The 
y3 edit output may depict how the clips would appear in presentation after editing is 

C= complete. To view the information, a reference element may be selected and 

'if 1 5 drug/dropped or cut/pasted onto the monitor window. However, such these actions do 
yS not necessarily result in the reference element being removed from its current place on 

* ~ the GUI, but simply allow the information to appear on the monitor window. 

Similarly, the GUI in an editing processing system may also be used to depict or 
Q play the time based stream of information (referred to as a "capture output"), as well as 

zl 20 other characteristics of the information that is being acquired. The capture output sets 
D forth the current frames that are being received and read, such as newly acquired 

information that is present in a proxy, i.e. buffer, prior to being written into storage. 
During the capture mode, the monitor window 246 may exhibit the capture output of 
the time based information. By the dual use of the monitor window 246 for both edit 
25 output and capture output, the screen area is conserved and the user interface is 
simplified. 

A monitor window time ruler 260 depicts where the play-head 262 as it travels 
along the course of the clip. A variety of output controls 248 may be used to move the 
play-head to various positions on the clip. When the play-head is at the frame of 
30 interest, the pause control 258 may be activated and the frame edited. In this manner, 
the user may select particular portions of information and instruct the system to edit the 
selected portion, e.g. deleting the selected portion from storage. 

One set of control elements are output controls 248 that implement how the user 
chooses to view or listen to the information from the system. Output control elements 
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may direct the showing of the information on the monitor window 246, the playing of 
information through audio speakers, or output through other convenient devices in 
communication with the system. There are several types of output controls 248 that may 
be provided by the processing system. 
5 The output controls 248 may be used to move the play-head (i.e. pointer to 

indicate the location in the information available for use) to particular sections of the 
information stored in the storage. One such output control is a play control 250 that 
directs the system to play or show the information. Other kinds of output controls 
include a rewind control 252 and a forward control 254. The play-head may be jogged 

10 backward by the user depressing the rewind control 252 to shuttle the play-head in a 
reverse direction. The output of information is advanced by the user selecting the 
forward control 254 during playback to shift forward at faster than the normal play rate, 
e.g. double the speed. Also provided by the processing system may be a stop control 
256 for ceasing the output action and a pause control 258 for temporarily suspending 

1 5 the movement of the information from the system. In practice, the stop control 256 may 
be selected and then the rewind control 252 or forward control 254 selected to quickly 
skip forward or backwards through the footage. 

In one method of outputting the time based information through the use of 
output controls, the play control 250 is activated and then the rewind control 252 or 

20 forward control 254 chosen to move to a position that is a immediately prior to the 
appropriate position for capture. The stop control 256 is selected and the play control 
250 is reactivated. When the scene of interest is displayed or a particular portion of 
audio is played, the clip may be edited. 

Several additional output controls may also be employed for alternative ways of 

25 communicating the information. For example, a full screen mode control 252 may be 
provided to permit the presentation to be shown in a window that spans the entire area 
of the display. Also, some control elements may adjust the quality of playback. A 
preferences option with a playback tab (not shown), and an edit pull down menu 254 
are some optional quality controls. In addition, a sound control knob 256 functions to 

30 adjust the intensity of audio. 

A smooth motion control may be present to allow the user to create a smoother 
playback than would be typically expected. Usually, the bandwidth necessary to display 
digital video requires that visual quality be compromised in order to deliver a high 
number of frames per second (fps) during playback. Also, a better image control may be 
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included to provide a crisper playback and more detail from each frame displayed. Such 
option for a better image is often at the expense of motion quality. The number of 
frames per second that may be displayed is dependent on the processing power of the 
system, where higher power results in higher frame rate and playback that is less 
5 choppy. 

Further to the GUI used in an editing processing system, a time code 237 may 
also be provided to indicate the length of a clip. In addition, a space monitor element 
234 may show the amount of storage space remaining or amount of space consumed by 
stored information. 

10 Still other control elements may receive reference elements in order to 

manipulate the references in forming segments of a presentation. The user may input 
commands to select and relocate reference elements 236 or portion(s) thereof, to the 
y3 several windows and portions on the GUI, such as a shelf 238, a video time line 240, an 

r : audio time line 242 or other portion of the display. One typical way for a user to select a 

J3 15 reference element is by pointing a cursor on the reference element and clicking on it, 
such as via a mouse or keyboard stroke. However, a user may use other tools for 
?y selection such as menu choices, voice instructions, etc. A selected reference may be 

H indicated on the GUI by a highlighted border 354 around the reference. 

fi In one embodiment, a reference element is moved to various portions of the 

^ 20 display by moving a pointer 356, e.g. cursor, over the reference element and dragging 
/J the reference element to another portion of the display. The reference element is 

dropped over the intended spot on the GUI by releasing the pointer from the reference 
element. 

Another means to move a reference element is though a cut and paste command. 
25 The reference element is selected and is made to disappear from the screen by 
activating a cut command. Activation of the cut command may occur by the user 
choosing a menu option, keyboard stroke, etc. An intended place for the relocation is 
selected and a paste command activated to prompt the reference element to reappear in 
the selected place. 

30 In addition, other functionalities may be provided by the processing system to 

relocate the reference element on the GUI, such as mouse clicks, keyboard commands, 
menu options, various types of control elements, etc. 

The reference to each clip that is to be included in the presentation is usually 
positioned in the presentation by arranging the reference element(s) along a time line 
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240, 242 in the order that the associated information will appear in the presentation. The 
graphical representation of a visual time line 240 is for adding visual information, such 
as the time based stream of information in the form of full motion images, e.g. video 
and movies, motion graphics, e.g. animation, as well as still images, e.g. text, pictures, 
5 combinations thereof, and the like. The graphical representation of an audio time line 
242 is for including audio information, such as music, sound effects, and 
voice/narration, etc. to the presentation. A user may opt for the audio synch time line to 
automatically display the references for each incoming stream of information. For 
example, the user may choose a preference control from a menu and select to have the 
10 references appear in the audio synch time line rather than the shelf portion of the user 
interface. The data corresponding to the time lines is stored in presentation storage area 
within the system. 

Often, the processing system includes functionality to display and use only a 
single visual time line 240 which is capable of handling any type of edit feature and 

15 other visual information. The visual time line may include visual reference elements 
with any combination of the edit features to be included in the presentation. Where it is 
desired for an edit feature it be nested in another reference (i.e. clip), the reference 
element may be split into two reference elements and the reference element having the 
edit feature is positioned between the two on the time line. A time code 237 may be 

20 provided that indicates the length of a clip associated with the reference. Furthermore, 
the time line may include the total time 358 of the presentation. The time line also may 
have a scroll bar 360 to move up and down the length of the time line, so that the time 
line is not limited by the space provided by the display. 

Edit features may be conveniently added to a presentation by use of an edit box 

25 268 that includes various editing control options, such as titles control 270, transitions 
control 300, music control 330 and sound control 350. Specific sub- windows having 
edit controls may be included for the various edit features contained in the edit box. In 
one embodiment, a sub-window pops-up on the GUI by activating an edit control 
option, e.g. by a mouse click. The sub-windows may also be retrieved by selecting an 

30 existing reference that has an edit feature. Upon selecting such as reference, the sub- 
window to the reference's edit feature automatically appears on the screen for use in 
further editing. These pop-up windows save screen space and create an easy editing 
environment for the user. 
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Title control 270 includes instructions for adding text, such as headings, labels 
or captions to a clip. Selection of the title control 270 opens a title window 274 as 
shown in one embodiment in Figure 6B. A transition time ruler 276 may be provided to 
show the length of the selected clip and the position of the play-head 278. The user may 
5 navigate the play-head to various frames in the clip by positioning a cursor over the 
play-head through use of a mouse, or using other cursor positioning means. The mouse 
may be used by clicking and holding down on it and sliding the mouse to the left or 
right along the title time-line relative to the desired location of the play-head. 

The title window 274 frequently also includes one or more text field(s) 280 for 
10 describing the desired text. A position menu 282 permits the user to choose where and 
how the text is to appear on the frame. For example, the text may be a centered title, 
appear to bounce through the length of the frame, centered with multiple text lines, 
y0 centered as a title, letters flying across the frame, words flying, music video, rolling 

Cj center credits, rolling credits, scroll with a pause, scrolling as a block, typewriter, and 

15 the like. A font menu 288 may be used to dictate the character type. An "over black" 
control 284 may be used to insert an additional black frame at the play-head position 
\ ^ and the title over the black frame. An example of a reference element with text over 

H black 285, having the title, "THE END" is shown on time line 240. On the other hand, a 

□ "color" control 286 may be used to overlay the text over the current frame. The text 
zl 20 may be shown in a preview window 290 by selecting a preview control 292. Preview 

□ simulates the edited reference without the edited version being actually saved in 
storage. An add control 294 may be provided for storing the reference with the title. 
The reference with edit feature 296 may then be shown on the monitor window 246. 
The initial clip 01 may be split into the reference with edit feature 296 and remaining 

25 portion reference 298, as shown on video time line 240. 

A transition control 300 has instructions for inserting an effect between clips, 
where selection of the transition control 300 opens a transition window 302, as depicted 
in Figure 6C. A transition effects menu 304 permits the user to choose from any one of 
numerous transition options. Examples of transitions are cross dissolve, cross dissolve 

30 slow, fade in, fade out, overlap, push right, slowly push right, scale down, slowly scale 
down, and the like. The frame location of the play-head 314 along transition time ruler 
316 may be positioned by the same or similar manner as described above for 
introducing titles in a clip. The transition is typically inserted as a frame immediately 
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after the play-head position by reference element with transition 352 and then appears 
on visual time line 240. 

Furthermore, the exemplary edit box 268 shown in Figure 6A includes music 
control 330 for incorporating music tracks to the presentation where the music may be 
recorded by controls in the music control window. Sound control 350 in the edit box 
268 is for including sound effect to portions of the presentation. Usually, the sound 
effects are prerecorded and stored within the processing system. Some examples of 
sounds are cat meow, dog bark, crickets, crowd applause, trumpets fanfare, wild laugh, 
water lapping, whistle, thunder, horn honk and footsteps. 

The various tasks that control the edit and storage of information and reference 
data are usually performed by a process manager. A process manager 66 is shown in 
Figure 2 coupled to storage 30 and processor 26 through bus 22. During the capturing 
of information, process manager 66 may allocate space on the storage for keeping a 
new information file. During the edit process, the user may make use of the tools 
described above in a variety of ways to form the desired presentation. 

The process manager receives selection commands, such as from a user input or 
software instructions, and responds by selecting the appropriate information and/or 
reference data, which may be an entire file of information/data or portions thereof. 
Typically, references are selected, viewed, optionally modified with edit features and 
transferred to a position on the time line, such as by drag/drop or cut/paste mechanisms. 
For example, a reference may be selected from the monitor window of an edit sub- 
window and either cut and pasted into the time line or drug from the monitor window 
and dropped into the time line. At any time, particular references may be removed from 
the presentation and returned to the shelf, also through the drag and drop or cut and 
paste tools. Where a reference is removed, the references remaining in the presentation 
shift over in the time line to create a seamless continuation of the presentation. 

There are numerous editing tasks that are performed by the process manager, 
some of which are described in the examples below. The exemplary editing methods so 
described may also be applied to adding any type of editing features to the presentation, 
such as text, transitions, audio, graphics, etc. 

(1) Cut and Paste Editing Example 

For one method of cutting or copying a portion of a clip during the processing of 
a presentation a first reference is initially provided, such as reference Al corresponding 
to clip A (0 to 50). Next, the processing manager copies the reference Al to create a 
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duplicate reference A2 (0 to 50). The process manager changes the reference data of 
either the original or copied reference to reflect edit command requests where the in 
point and out point of reference A2 are modified to (0 to 20). The process manager 
places modified reference A2 onto a paste hoard for inserting into a presentation. The 

5 reference data that is present in the paste board may be inserted at any point in the 
presentation along the time line in presentation storage area. If the reference is to be 
placed prior to or after another reference, already present in the presentation storage 
area, or if there are no other references provided in the presentation storage area, the 
reference is simply pasted into the storage area. On the other hand, where the reference 

10 is to be nested within another reference, third reference Bl (0 to 30), the reference Bl 
(0 to 30) is first split at the point where insertion is to occur (unit 15) by copying the 
reference Bl (0 to 30) to create reference B2 (0 to 30). The in and out points are 
changed so that the reference Bl (0 to 15) corresponds to the first clip portion prior to 
the inserted reference A2 and the reference B2 (16 to 30) corresponds to the second clip 

15 portion following reference A2. The modified reference A is then pasted between the 
two reference B's in the presentation storage area. 

In subsequent steps, the cut and paste process may be undone to retrieve the 
original data. For example, reference A2 may be deleted in response to a deletion 
command and reference Al is preserved. The continuum time line between reference 

20 Bl and B2 is also restored such that there is no gap where reference A2 had previously 
been positioned. Although referenced Bl and B2 may remain split, the two references 
run seamlessly to appear as though there is a single reference to the clip. 
(2) Adding Edit Features Example 

In one method of adding edit features to a clip during the processing of a 
25 presentation, a reference Al corresponding to clip A (0 to 50) is provided. The 
reference Al is copied to create a duplicate reference A2 (0 to 50). Usually, selected 
edit features may be added to either the beginning (first frame) or ending portions (last 
frame) of the clip represented by the original or the copy reference. Where the edit 
feature is intended for the beginning or ending unit(s) of a reference, the feature is 
30 simply inserted in the reference and the reference having the edit feature is placed in 
the presentation storage area. Movement of the reference may be through the cut and 
paste methods described above or by a user command to drag and drop the reference in 
place. 
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In the alternative, text may be intended for a middle portion of a clip. The 
reference A2 (0 to 50), that is targeted for text editing is split at the location (between 
unit 25 and 26) for the text by copying to the reference and forming reference A2(a) (0 
to 50), and reference A2(b) (0 to 50). The in and out points of the references are 
5 modified to reflect the split to reference A2(a) (0 to 25) and reference A2(b) (26 to 50). 
The text may be added to the beginning unit(s) or ending unit(s) of the split reference 
152 and the edited reference added to the presentation storage area 154. The text may 
be added to the last unit of reference A2(a) at 25 and may additionally be included in 
any sequential units previous to 26 without skipping any units in the timeline. It is also 
10 possible to add the text to reference A2(b) starting at unit 26, and to any sequentially 
following units without skipping any such units. 

In addition, where a user does not need a segment of information for the 
35 presentation, a clip or portion of a clip may be removed from storage. Entire clips or 

?] portions of clips may be deleted by selecting the reference data and dragging it to the 

«3 15 trash icon control 370 on the GUI. After depositing the reference data in the trash, the 
Jl user may activate a delete control. For example, the user may select an "empty trash" 

^ e.g. from a pull down menu. In still other embodiments, the user may activate a "cut" 

H command, e.g. from a pull down menu, to have the selected data placed on a paste 

S{ board. The user may then send delete instructions to the processing system and the data 

2 20 as well as corresponding information is deleted. In another embodiment, an 
{E intermediary trash bin or paste board is not used and the user simply elects to delete 

selected reference data. Other methods for the user to send selection, deletion and trash 
commands are also possible and intended by the present invention. 

Where the system receives such trash and/or delete command from a user input 
25 or software instruction or other means, the process manager 44 may respond by deleting 
the relevant information and references thereof. Furthermore, where the selected data is 
cut and placed onto a paste board, the process manager may respond to a delete 
command by deleting the data from the paste board and as well as its associated 
information. 

30 An optional trash depository 58 may be provided in storage 30, as shown in 

Figure 2, to temporarily hold references, reference data, the time based stream of 
information, or any portions thereof. The contents of the trash depository may be 
accessed, manipulated, or removed by the processing system. Often, the trash 
depository 58 serves as a holding bin for unwanted data prior to its deletion from 
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storage. By dropping the data into the trash depository, the user may continue with 
uninterrupted editing and later empty the trash, i.e., delete the contents and any 
associated information, at a convenient time. At any time while the reference data is in 
the trash depository, the processing system may receive a cancel command, such as 
5 from the user or other means. In the case of a cancel command, the process manager 
returns the reference data back to its original location and restores the initial reference, 
e.g. in and out points, rather than deleting the reference and the information. 

The process manager may perform such delete action by directly and 
permanently eliminating the selected information (and associated reference data) from 
10 storage. This direct elimination may take place by the process manager copying over 
the relevant file. If there is any information occurring prior to or after the eliminated 
section, the remaining information appears to be continuous and uninterrupted by the 
^3 copy over the excluded section. In the alternative, the deletion may also take place by 

changing a virtual file allocation table (VFAT, FAT or FAT32) to indicate that the 
yp 1 5 space that has been used by that file is available for reuse by another file. When new 
files are written to that space, the deleted data is overwritten and thus, permanently 
5 w eliminated. Prior to this overwriting, it may be possible for the deletion to be reversible. 

In one embodiment, direct elimination of information occurs where the 
S beginning portion or middle portion of a clip in a file is selected for deletion. This 

Ul 20 elimination may occur by various mechanisms. In one embodiment, the system may 
O copy the portion that is to be preserved into a new file and the old file may be removed. 

In another instance, the system may simply shift over the preserved data to replace the 
eliminated data. For example, where units 0-30 are to be eliminated from a clip of 0-40, 
units 31 to 40 may be moved into the space previously occupied by 0-30, thus creating 
25 a clip 0-9. In this manner the header information that may be present in the beginning of 
a clip may also be preserved. 

Where a tail end of a clip in a file is to be deleted, the clip may be merely 
truncated such that the ending portion is marked as extra space that is available for 
reuse. Thus, the beginning header information is still present in the file. Furthermore, in 
30 order to erase all of a file that is selected for elimination, the deletion may take place by 
merely marking the entire file as deleted and available for reuse as described above. 

In any case, the process manager may also be triggered to perform several 
relevant routines upon receipt of the delete command. In one embodiment, a reference 
check method is performed when the system receives a selected information command 
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and a delete command. The process manager responds to the delete command by 
checking in the reference database to determine if a single or no reference data to the 
selected information exists. Where the process manager concludes that more than one 
reference data set to the selected information exists, the process manager fails to 
5 eliminate the selected information from storage. Rather than deleting the information 
from storage, only the selected reference data is deleted. On the other hand, if the 
process manager establishes that more than one reference data set to the selected 
information does not exist, the process manager deletes the selected reference data 
contained within a reference or portion of a reference, if any are such data present in the 

1 0 storage, and also deletes the selected information from storage. 

The presentation, as edited, may be saved in a variety of formats, such as a 
QuickTime file or other formats suitable for transfer via e-mail, web pages, CD-ROM, 
etc. The presentation may be accessed at a later time for additional editing or may be 
transferred from the processing system, such as conveyed to an external player. 

15 At times, an edited version of the time based stream of information may be 

produced by the system and stored. The time based stream of information that 
corresponds to an edited reference is copied into storage. Then, either the original or 
copy of the information is modified to reflect the edited reference. Thus, the storage 
includes both the original time based stream of information and edited version of the 

20 information. 

The total amount of storage space available for new files is determined by a 
space monitor 68. Space monitor 68 is coupled to storage 30 through bus 22. When a 
new file is created to hold information, the space monitor ascertains the amount of 
space that is utilized by the new file and subtracts this consumed space to recalculate 

25 the remaining amount of space in storage. 

In one configuration of processing system 12, a means for communicating with 
an information source that has a time based stream of information is provided. For 
example, bus 22 additionally couples detector 24 to the processor 26. The processing 
system may detect a connection between the processing system and the information 

30 source through detector 24. The detector 24 performs an inquiry as to whether an 
information source is in communication with the processing system. The detector 
determines a connection with the information source by sensing particular signals, such 
as electrical current output, that are released from the information source, and usually 
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when the source is energized, i.e. "turned-on." The detector may be further configured 
to conduct periodic checks for communication with the information source. 

Means are provided for capturing the time based stream of information from the 
information source, such as a capture card or other capture mechanisms. The transfer 
5 rate of the information entering the system is usually the same or substantially the same 
speed that the information runs prior to entering the system, such as the rates 
established by the video standards of various regions. For example, movie film usually 
runs with a speed of 24 fps. Some video follows the NTSC (system used in North 
America and Japan), where the video speed is 30 fps. 
10 The information may be captured and written into storage by various 

mechanisms. One capture mechanism is a sequential-type method in which an 
application procedure is executed when the processing system is in capture mode. In 
y3 one stage of the application procedure, signals from an information source are captured, 

Zl The system reads the information, shows the information on the display and writes the 

15 information to a storage medium. However, when control elements are activated, the 
yg system moves away from the capture step and switches to another stage that includes a 

' " process control commands loop for handling the control instructions. The CPU of a 

H system employing such a sequential-type method is dedicated to processing the control 

q commands. While application procedure is in the process control command loop, the 

!L? 20 capturing operation ceases and the processing system stops communicating with the 
O information source. During this period, the information that is released from the 

information source does not get loaded into the system. Rather, the processing of 
control commands continues through to completion, at which time, the capturing step 
may continue. 

25 Another capture mechanism, according to the present invention, is a low-level 

interrupt-type method, which effectively permits intermittent processing of control 
element commands while, at the same time, loading the time-based information. With 
the interrupt-type method, in addition to the capture application, a user may run any 
software applications, e.g. network communication protocols, such as Appletalk, 

30 Internet or e-mail applications, word processing applications, etc. 

An interrupt-type method includes an application procedure that may be 
executed when the processing system is in capture mode and when it is not in capture 
mode, e.g. when the system is in edit mode. Where the processing system is operating 
in a non-capture mode, the application procedure conducts normal operations, such as 
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processing of control commands from control elements. However, when the capture 
mode is indicated, the interrupt procedure fires the appropriate capture tasks and causes 
a temporary suspension of the application procedure in an asynchronous manner. Thus, 
the processing of control commands proceeds until the capture process interrupts the 
5 other application tasks. 

This interrupt procedure may be performed at periodic intervals and are certain 
to iterate at regular times, despite the activities, e.g. processing of control commands, of 
the application procedure. The interrupt procedure includes a quick reading of incoming 
information, copying the information into a proxy, outputting the information and 

10 writing it into storage where the information may be further edited by use of the control 
elements. The interrupt procedure is advantageous in that the process is very quickly 
performed and rapidly repeats. The time for performance is less than the transfer rate of 
the information entering the system. 

The repetitions of the interrupt procedure occur at a rate that is substantially 

15 similar to or the same as the transfer rate. Thus, all of the information may be read and 
stored while still processing control commands in-between each of the iterations of the 
interrupt procedure. As a result, the user does not recognize the interruptions and the 
capture process as well as other instructions appear to be simultaneously executed by 
the system. Moreover, the interrupt procedure guarantees that information is not 

20 omitted from the reading step because the iterations of the interrupt procedure and 
length of time to perform are timed to correspond with the rate of information transfer 
from the information source. 

For transfer of NTSC video, the interrupt procedure iterations are typically at 
least every 1/29 to 1/30 second and more usually 1/29.97 second to 1/30 second. For 

25 PAL systems (systems used in parts of Europe, China and Hong Kong), the frame rate 
is 25 fps and thus interrupt procedure is substantially at least every 1/25 second. In 
general, the interrupt procedure should iterate close to the transfer rate or slightly faster 
than the transfer rate, e.g. within at least 1/1 0th second of the transfer rate and more 
typically within 1/1 00th second. The iterations should not be slower than the transfer 

30 rate and the iterations are usually at the rate that corresponds to the frame rate of video 
standard. 

Usually, the processing system according to the present invention, outputs the 
information, e.g. depicts the information on the display at a rate (the "output rate") that 
is substantially similar to the rate in which each unit of the information, e.g. frame of 
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video, is arriving from the information source (the "transfer rate"). In contrast with 
prior systems, where low quality displaying (i.e. considerably lower than the transfer 
rate, such as 30 to 70% of the transfer rate) is used to devote more of the system to 
capture, the present invention may use high quality (i.e. the same or substantially 
5 similar to the transfer rate, such as greater than 70% and more usually 93 to 100% of 
the transfer rate) because all of the information is certain to be stored by the interrupt 
procedure. The speed at which the information is displayed depends on numerous 
factors, such as side operations being simultaneously performed by the system, e.g. 
network access and CD ROM detection, image quality, e.g. based on decompression 

10 schemes employed, and processor speed or graphic acceleration. For example, where 
the transfer rate is about 30 frames per second (fps), the corresponding output rate of 
the present invention is usually 29 to 30 fps. The transfer rate depends on the particular 
transfer rate standard employed, as described in detail below. The resulting capture 
output is of high quality that is similar to the quality of the information residing in the 

1 5 information source. 

During the capture process, the time based stream of information is written into 
storage to be referenced while creating the presentation. For convenience, the stored 
information may be compressed to conserve storage space. Generally, compression 
formats are either high efficiency or low efficiency and either loss-less or lossy. Lossy 

20 compression schemes are characterized by components of an original image being 
absent from a reconstructed image after a compression-decompression cycle, whereas 
loss-less schemes do not drop any information. 

Low compression schemes (i.e., those that do not provide significant 
compression ratios) that may be used include joint photographic experts group (JPEG) 

25 compression schemes that use Fourier analysis-based methods, such as the discrete 
cosine transform, to compress data; graphics interchange format (GIF) compression 
schemes, which use LZW algorithms; bitmapped image compression schemes and 
tagged image file format (TIFF) compression schemes, etc. Alternatively, high 
efficiency compression schemes, such as wavelet, motion wavelet, Motion Picture 

30 Experts Group (MPEG) motion JPEG, Lempel Ziv and/or fractal compression schemes, 
etc., may be employed. Use of high efficiency schemes may be preferred where storage 
space is limited. For example, wavelet compression schemes may be 20 times more 
efficient than JPEG compression schemes, providing for a more detailed representation 
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of the data at the same bit budget. Once the time based information is captured and 
stored, it may be referenced for editing and generating a presentation. 

In addition to the features described above, the processing system may further 
include alternative aspects that enhance ease and convenience of use. For example, 
5 referring again to Figure 2, the information source 80 may be controlled through the 
processing system 12. Source controls operate the information source 80, usually where 
no digital converter 82 is employed as an intermediate between the information source 
and processing system. The source controls may be presented on the GUI. In one 
embodiment, the output controls 248, as shown in Figure 7A also serve as the source 
10 controls to direct the monitor window views at one time and send action commands to 
the information source 80 at another time. Thus, the source controls may include rewind 
control 252, forward control 254, play control 250, pause control 258 and stop control 
& 256. 

Zj Software Overview 

Jf 15 Various software components, e.g. application program, may be provided within 

or in communication with the system that cause the processor to execute the numerous 
*S* methods employed in creating the presentation. Figure 8 is a block diagram of a 

^ machine-readable medium storing executable code and/or other data to provide one or a 

Q combination of operations, e.g. mechanisms for collecting and manipulating the time 

%^ 20 based stream of information, according to one embodiment of the invention. The 
3 machine-readable storage medium 400 represents one or a combination of various types 

of media/devices for storing machine-readable data, which may include machine- 
executable code or routines. As such, the machine-readable storage medium 400 could 
include, but is not limited to one or a combination of a magnetic storage space, 
25 magneto-optical storage, tape, optical storage, dynamic random access memory, static 
RAM, flash memory, etc. Various subroutines may also be provided. These subroutines 
may be parts of main routines or added as plug-ins or Active-X controls. 

The machine readable storage medium 400 is shown storing a GUI routine 402, 
which, when executed, provides a layered GUI for display of control elements, such as 
30 the GUI 230 that may be used in an editing processing system shown in Figures 7A- 
7C. The GUI routine 402 includes a number of mechanisms for playing or otherwise 
displaying the time based stream of information. A subroutine for accessing a graphic 
file The presentation, as edited, may be saved in a variety of formats, such as a 
QuickTime file or other formats suitable for transfer via e-mail, web pages, CD-ROM, 
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etc. The presentation may be accessed at a later time for additional editing or may be 
transferred from the processing system, such as conveyed to an external player. 

A subroutine for accessing the graphic file 404 is provided, as described in 
regards to Figures 3 to 5. Usually the layers of the graphic file are read in hierarchical 
5 order. Furthermore, a present control elements subroutine 408 permits control elements 
to be visible on the display as determined by control objects in the graphic file. Another 
subroutine may be for launching an edit application program that displays the user 
interface and a further subroutine to manipulate the control objects for altering the 
control element attributes. 
10 The medium 400 also has a process control command routine 418 that directs 

the carrying out of various commands, such as for manipulating information and 
constructing a presentation. The control commands may be generated by activation of 
S control elements by the user. Upon receipt of the control commands, the process control 

Z[ command 418 may send instructions to the GUI Routine 402 to specify particular 

15 information to be presented or manner of presenting information. The process control 

yC command 418 routine may also send the commands to a process management routine, 

ft « 

such as an add edit feature command, undo command, delete command, select 
information command, cancel delete command, and the like, as described below. 
3 The medium also has a process management routine 410 that may have various 

!fj 20 subroutines for conducting various tasks related to editing and carrying out instructions 
O from the process control command routine 418. One such subroutine is a 

cut/copy/paste subroutine 412, which when executed, references or portions thereof are 
cut, copied and/or pasted into a presentation. The process management routine 410 may 
be also configured for conducting an add edit feature subroutine 416 to include 
25 particular edit features to references for use in a presentation. An undo edit subroutine 
414 may optionally be included whereby certain edit tasks may be undone to reclaim 
the unmodified version of data 

Other software components may include an operating system 420 and a 
presentation transfer routine 422 that facilitates communication of information from the 
30 processing system, such as via external player 84. Various modes of transfer are 
described above with reference to the external player 84 in Figure 2. 

The software components may be provided in as a series of computer readable 
instructions that may be embodied as data signals in a carrier wave. When the 
instructions are executed, they cause a processor, such as through a browser to perform 
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the capture and editing steps as described. For example, the instructions may cause a 
processor to communicate with an information source, provide capture information 
from the time based stream of information on a first portion of a display, provide 
process information for constructing the presentation on a second portion of the display; 
5 and provide at least one enabled control element. Such instructions may be presented to 
the processor by various mechanisms, such as a plug-in, ActiveX control, through use 
of an applications service provided or a network, etc. 

The present invention has been described above in varied detail by reference to 
particular embodiments and figures. However, these specifics should not be construed 

10 as limitations on the scope of the invention, but merely as illustrations of some of the 
presently preferred embodiments. It is to be further understood that other modifications 
or substitutions may be made to the described information transfer system as well as 
methods of its use without departing from the broad scope of the invention. Therefore, 
the following claims and their legal equivalents should determine the scope of the 

1 5 invention. 
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1 CLAIMS 

2 What is claimed is: 



3 1 . A method for producing a graphical user interface, the method comprising: 

4 storing a graphic file having at least one control object, each control 

5 object in a separate layer, and 

6 launching an application program to access the graphic file and to 

7 display a control element on the graphical user interface, the control 

8 element having at least one attribute dictated by one of the control 

9 objects. 

C3 10 2. The method of claim 1, wherein the graphic file has a first control object in a 
gi 1 1 layer dictating one attribute of the control element and a second control object in 

2 12 a separate layer dictating another attribute of the control element. 

J: 13 3. The method of claim 2, wherein the first control object and the second control 
s 14 object share a common name attribute. 

O 15 4. The method of claim 2, wherein the layer of the first control object is grouped 
□ 16 with the layer of the second control object. 

17 5. The method of claim 1, wherein the graphic file is editable and the at least one 

1 8 control object may be added, deleted or altered. 

19 6. The method of claim 1, wherein the control element is an edit control to 

20 manipulate a time-based stream of information. 

21 7. The method of claim 1 , wherein the attribute is an appearance, location or size. 

22 8. The method of claim 1, wherein the attribute is the element type, state, function 

23 or behavior in a particular environment. 
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1 9. A computer system comprising: 

2 a storage; 

3 a display device; and 

4 a processor for: 

5 storing a graphic file having at least one control object, each 

6 control object in a separate layer, and 

7 launching an application program to access the graphic file and to 

8 display a control element on the graphical user interface, the 

9 control element having at least one attribute dictated by one of 
1 0 the control obj ects. 

O 

J? 11 10. The system of claim 9, wherein the graphic file has a first control object in a 

%I 12 layer dictating one attribute of the control element and a second control object in 

ffi 13 a separate layer dictating another attribute of the control element. 

g 14 11. The system of claim 10, wherein the first control object and the second control 

~ 15 object share a common name attribute. 

JL? 16 12. The system of claim 10, wherein the layer of the first control object is grouped 

D 17 with the layer of the second control object. 

18 13. The system of claim 9, wherein the graphic file is editable and the at least one 

19 control object may be added, deleted or altered. 

20 14. The system of claim 9, wherein the control element is an edit control to 

2 1 manipulate a time-based stream of information. 

22 15. The system of claim 9, wherein the attribute is an appearance, location or size. 

23 16. The system of claim 9, wherein the attribute is the element type, state, function 

24 or behavior in a particular environment. 

25 17. A system for producing a graphical user interface, comprising: 
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1 means for storing a graphic file having at least one control object, each 

2 control object in a separate layer; 

3 means for launching an application program to access the graphic file 

4 and to display a control element on the graphical user interface, the 

5 control element having at least one attribute dictated by one of the 

6 control objects. 

7 18. The system of claim 17, wherein the graphic file has a first control object in a 

8 layer dictating one attribute of the control element and a second control object in 

9 a separate layer dictating another attribute of the control element. 

10 19. The system of claim 18, wherein the first control object and the second control 

J3 1 1 object share a common name attribute. 

y3 12 20. The system of claim 18, wherein the layer of the first control object is grouped 

13 with the layer of the second control object. 

y, 14 21. The system of claim 17, wherein the graphic file is editable and the at least one 

y 1 5 control object may be added, deleted or altered. 



16 22. The system of claim 17, where in the control element is an edit control to 

17 manipulate a time-based stream of information. 

18 23. The system of claim 17, wherein the attribute is an appearance, location or size. 

19 24. The system of claim 17, wherein the attribute is the element type, state, function 

20 or behavior in a particular environment. 

21 25. A computer readable medium having stored therein a plurality of sequences of 

22 executable instructions, which, when executed by a computer system for 

23 producing a graphical user interface, cause the processor to: 

24 store a graphic file having at least one control object, each control object 

25 in a separate layer; 



Application 



51 



004860.P2475 



1 launch an application program to access the graphic file and to display a 

2 control element on the graphical user interface, the control element 

3 having at least one attribute dictated by one of the control objects. 

4 26. The computer readable medium of claim 25, wherein the graphic file has a first 

5 control object in a layer dictating one attribute of the control element and a 

6 second control object in a separate layer dictating another attribute of the control 

7 element. 

8 27. The computer readable medium of claim 26, wherein the first control 

9 object and the second control object share a common name attribute. 

28. The computer readable medium of claim 26, wherein the layer of the first 
control object is grouped with the layer of the second control object. 

29. The computer readable medium of claim 27, wherein the graphic file is editable 
and the at least one control object may be added, deleted or altered. 

30. The computer readable medium of claim 25, wherein the control element is an 
edit control to manipulate a time-based stream of information. 

31. The computer readable medium of claim 25, wherein the attribute is an 
appearance, location or size. 

18 32. The computer readable medium of claim 25, wherein the attribute is the element 

19 type, state, function or behavior in a particular environment. 
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ABSTRACT 

A method of creating a graphical user interface having control(s) to direct any 
task through a computer system is provided. One application of the user interface is for 
manipulating a time based stream of information to create a presentation. The methods 
use a layered graphic file that is accessed by an application program to generate the user 
interface. Each layer of the graphic file contains a control object to an element of the 
user interface. The control objects are capable of being revised in order to make 
alterations to the user interface. Other aspects of the present invention relating to the 
graphical user interface are also described. 
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As a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below, next to my name. 

I believe I am the original, first, and sole inventor (if only one name is listed below) or an original, first, and 
joint inventor (if plural names are listed below) of the subject matter which is claimed and for which a 
patent is sought on the invention entitled 

LAYERED GRAPHICAL USER INTERFACE 



the specification of which 

X is attached hereto. 

was filed on as 

United States Application Number 

or PCT International Application Number 

and was amended on „ - 

(if applicable) 

I hereby state that I have reviewed and understand the contents of the above-identified specification, 
including the claim(s), as amended by any amendment referred to above. I do not know and do not 
b e li eve that the claimed invention was ever known or used in the United States of America before my 
invention thereof, or patented or described in any printed publication in any country before my invention 
thereof or more than one year prior to this application, that the same was not in public use or on sale in 
the United States of America more than one year prior to this application, and that the invention has not 
been patented or made the subject of an inventor's certificate issued before the date of this application in 
any country foreign to the United States of America on an application filed by me or my legal 
representatives or assigns more than twelve months (for a utility patent application) or six months (for a 
design patent application) prior to this application. 

I acknowledge the duty to disclose all information known to me to be material to patentability as 
defined in Title 37, Code of Federal Regulations, Section 1.56. 

I hereby claim foreign priority benefits under Title 35, United States Code, Section 1 19(a)-(d), of any 
foreign application(s) for patent or inventor's certificate listed below and have also identified below 
any foreign application for patent or inventor's certificate having a filing date before that of the 
application on which priority is claimed: 
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Prior Foreign Application(s) 



Priority 
Claimed 



(Number) (Country) (Day/Month/Year Filed) Yes No 



(Number) (Country) (Day/Month/Year Filed) Yes No 



(Number) (Country) {Day/Month/Year Filed) Yes No 

1 hereby claim the benefit under title 35, United States Code, Section 1 19(e) of any United States 
provisional application(s) listed below: 



(Application Number) Filing Date 



(Application Number) Filing Date 



I hereby claim the benefit under Title 35, United States Code, Section 120 of any United States 
application(s) listed below and, insofar as the subject matter of each of the claims of this application 
is not disclosed in the prior United States application in the manner provided by the first paragraph 
of Title 35, United States Code, Section 112, I acknowledge the duty to disclose all information 
known to me to be material to patentability as defined in Title 37, Code of Federal Regulations, 
Section 1 .56 which became available between the filing date of the prior application and the national 
or PCT international filing date of this application: 



(Application Number) Filing Date (Status - patented, 

pending, abandoned) 



(Application Number) Filing Date (Status - patented, 

pending, abandoned) 

I hereby appoint the persons listed on Appendix A hereto (which is incorporated by reference and a 
part of this document) as my respective patent attorneys and patent agents, with full power of 
substitution and revocation, to prosecute this application and to transact all business in the Patent 
and Trademark Office connected herewith. 

Send correspondence to Lisa N. Benado , BLAKELY, SOKOLOFF, TAYLOR & 

(Name of Attorney or Agent) 
ZAFMAN LLP, 12400 Wilshire Boulevard 7th Floor, Los Angeles, California 90025 and direct 
telephone calls to Lisa N. Benado . (408) 720-8598. 

(Name of Attorney or Agent) 
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I hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these 
statements were made with the knowledge that willful false statements and the like so made 
are punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United 
States Code and that such willful false statements may jeopardize the validity of the 
application or any patent issued thereon. 



Full Name of Sole/First Inventor Glenn Reid 



Inventor's Signature Date 

Residence Citizenship . 



(City, State) (Country) 
Post Office Address 



Full Name of Second/Joint Inventor Priscilia Shih 



Inventor's Signature Date 

Residence Citizenship . 



(City, State) (Country) 
Post Office Address 



Full Name of Third/Joint Inventor 

Inventor's Signature Date 

Residence Citizenship 

(City, State) (Country) 

Post Office Address 



Full Name of Fourth/Joint Inventor 



Inventor's Signature Date 

Residence Citizenship . 



(City, State) (Country) 
Post Office Address 
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APPENDIX A 



William E. Alford, Reg. No. 37,764; Farzad E. Amini, Reg. No. P42,261; Aioysius T. C. AuYeung, Reg. No. 
35,432; William Thomas Babbitt, Reg. No. 39,591; Carol F. Barry, Reg. No. 41,600; Jordan Michael 
Becker, Reg. No. 39,602; Bradley J. Bereznak, Reg. No. 33,474; Michae! A. Bernadicou, Reg. No. 35,934; 
Roger W. Blakely, Jr., Reg. No. 25,831; Gregory D. Caldwell, Reg. No. 39,926; Andrew C. Chen, Reg. No. 
43,544; Thomas M. Coester, Reg. No. 39,637; Dennis M. deGuzman, Reg. No. 41,702; Stephen M. De 
Klerk, under 37 C.F.R. § 10.9(b); Michael Anthony DeSanctis, Reg. No. 39,957; Daniel M. De Vos, Reg. 
No. 37,813; Robert Andrew Diehl, Reg. No. 40,992; Sanjeet Dutta, Reg, No. P46,145; Matthew C. Fagan, 
Reg. No, 37,542; Tarek N. Fahmi, Reg. No. 41,402; Paramita Ghosh, Reg. No. 42,806; James Y. Go, 
Reg. No. 40,621; James A. Henry, Reg. No. 41,064; Wiflmore F. Hoibrow III, Reg. No. P41,845; Sheryl 
Sue Holloway, Reg. No. 37,850; George W Hoover li, Reg. No. 32,992; Eric S. Hyman, Reg. No. 30,139; 
William W. Kidd, Reg. No. 31,772; Sang Hui Kim, Reg. No. 40,450; Eric T. King, Reg. No. 44,188; Kurt P. 
Leyendecker, Reg. No. 42,799; Michael J. Mallie, Reg. No. 36,591; Andre L. Marais, under 37 C.F.R. § 
10.9(b); Paul A. Mendonsa, Reg. No. 42,879; Chun M. Ng, Reg. No. 36,878; Thien T. Nguyen, Reg. No. 
43,835; Thinh V. Nguyen, Reg. No. 42,034; Dennis A. Nicholls, Reg. No. 42,036; Daniel E. Ovanezian, 
Reg. No. 41,236; Marina Portnova, Reg. No. P45,750; Babak Redjaian, Reg. No. 42,096; William F. 
Ryann, Reg. 44,313; James H. Salter, Reg. No. 35,668; William W. Schaai, Reg. No. 39,018; 
James C. Scheller, Reg. No. 31,195; Jeffrey Sam Smith, Reg. No. 39,377; Maria McCormack Sobrino, 
Reg. No. 31,639; Stanley W. Sokoloff, Reg. No. 25,128; Judith A. Szepesi, Reg. No. 39,393; Vincent P. 
Tassinari, Reg. No. 42,179; Edwin H. Taylor, Reg. No. 25,129; John F. Travis, Reg. No. 43,203; Joseph 
A. Twarowski, Reg. No. 42,191; Lester J. Vincent, Reg. No. 31,460; Glenn E. Von Tersch, Reg. No. 
41,364; John Patrick Ward, Reg. No. 40,216; Mark L. Watson, Reg. No. P46,322; Thomas C. Webster, 
Reg. No. P46,154; Charles T. J. Weigell, Reg. No. 43,398; Kirk D. Williams, Reg. No. 42,229; Steven D. 
Yates, Reg. No. 42,242; and Norman Zafman, Reg. No. 26,250; my patent attorneys, and Justin M. Dillon, 
Reg. No. 42,486; my patent agent, of BLAKELY, SOKOLOFF, TAYLOR & ZAFMAN LLP, with offices 
located at 12400 Wilshire Boulevard, 7th Floor, Los Angeles, California 90025, telephone (310) 207-3800, 
and James R. Thein, Reg. No. 31 ,710, my patent attorney. I also hereby appoint Albert P. Cefalo, Reg. 
No. 27,315; Mark Aaker, Reg. No. 32,667, Richard Liu, Reg. No. 34,377; Helene Plotka Workman, Reg. 
No. 35,981; and Edward W. Scott, IV, Reg. No. 36,000; my attorneys; of APPLE COMPUTER, INC., 
located at 1 Infinite Loop, MS: 3-PAT, Cupertino, California 95014, telephone (408)974-9453, will full 
power of substitution and revocation, to prosecute this application and to transact all business in the 
Patent and Trademark Office connected herewith. 
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APPENDIX B 



Title 37, Code of Federal Regulations, Section 1.56 
Duty to Disclose Information Material to Patentability 

(a) A patent by its very nature is affected with a public interest. The public interest is best served, 
and the most effective patent examination occurs when, at the time an application is being examined, the 
Office is aware of and evaluates the teachings of all information material to patentability. Each individual 
associated with the fifing and prosecution of a patent application has a duty of candor and good faith in 
dealing with the Office, which includes a duty to disclose to the Office all information known to that individual 
to be material to patentability as defined in this section. The duty to disclosure information exists with respect 
to each pending claim until the claim is cancelled or withdrawn from consideration, or the application becomes 
abandoned. Information material to the patentability of a claim that is cancelled or withdrawn from 
consideration need not be submitted if the information is not material to the patentability of any claim 
remaining under consideration in the application. There is no duty to submit information which is not material 
to the patentability of any existing claim. The duty to disclosure all information known to be material to 
patentability is deemed to be satisfied if all information known to be material to patentability of any claim 
issued in a patent was cited by the Office or submitted to the Office in the manner prescribed by §§1 .97(b)-(d) 
and 1.98. However, no patent will be granted on an application in connection with which fraud on the Office 
was practiced or attempted or the duty of disclosure was violated through bad faith or intentional misconduct. 
The Office encourages applicants to carefully examine: 

(1 ) Prior art cited in search reports of a foreign patent office in a counterpart application, and 

(2) The closest information over which individuals associated with the filing or prosecution of a 
patent application believe any pending claim patentably defines, to make sure that any material information 
contained therein is disclosed to the Office. 

(b) Under this section, information is material to patentability when it is not cumulative to 
information already of record or being made or record in the application, and 

(1) It establishes, by itself or in combination with other information, a prima facie case of 
unpatentability of a claim; or 

(2) It refutes, or is inconsistent with, a position the applicant takes in: 

(i) Opposing an argument of unpatentability relied on by the Office, or 

(ii) Asserting an argument of patentability. 

A prima facie case of unpatentability is established when the information compels a conclusion that a claim is 
unpatentable under the preponderance of evidence, burden-of-proof standard, giving each term in the claim 
its broadest reasonable construction consistent with the specification, and before any consideration is given to 
evidence which may be submitted in an attempt to establish a contrary conclusion of patentability. 

(c) Individuals associated with the filing or prosecution of a patent application within the 
meaning of this section are: 

(1) Each inventor named in the application; 

(2) Each attorney or agent who prepares or prosecutes the application; and 

(3) Every other person who is substantively involved in the preparation or prosecution of the 
application and who is associated with the inventor, with the assignee or with anyone to whom there is an 
obligation to assign the application. 

(d) Individuals other than the attorney, agent or inventor may comply with this section by 
disclosing information to the attorney, agent, or inventor. 
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